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ABSTRACT 


The  central  composite  designs  are  investigated  in  terms  of  their 
ability  to  detect  lack  of  fit  and  it  is  shown  that  measures  of  non- 
quadraticity  can  be  obtained  in  all  the  k  major  directions.  The 
geometrical  interpretation  of  this  is  explained.  The  relationship  of 
these  lack  of  fit  measures  to  the  estimation  of  power  transformations 
in  the  predictor  variables  is  also  explored. 
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SIGNIFICANCE  AND  EXPLANATION 


A  popular  and  useful  class  of  experimental  designs  for  fitting  a 
second  degree  equation  is  the  "central  composite"  class  which  consists  of 
two-level  factorials,  plus  axial  points,  plus  center  points.  These  are 
investigated  in  terms  of  their  ability  to  detect  lack  of  fit  and  it  is 
shown  that  measures  of  non-quadraticity  can  be  obtained  in  all  the 
k  major  directions.  The  geometrical  interpretation  of  this  is 
explained.  The  relationship  of  these  lack  of  fit  measures  to  the 
estimation  of  power  transformations  in  the  predictor  variables  is  also 
explored. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  authors  of  this  report. 
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MEASURES  OF  LACK  OF  FIT,  AND  TRANSFORMATION 
OF  PREDICTORS,  IN  COMPOSITE  DESIGNS* 

G.  E.  P.  Box  and  N.  R.  Draper 


1 .  INTRODUCTION 

In  the  course  of  a  response  surface  study  (see,  for  example.  Box, 

Hunter,  Hunter,  1979)  we  may  desire  to  fit  a  first  (all  -  0)  or  second 

•  J 

order  model 


relating  a  response  y  and  k  coded  predictor  variables  ,x2 . x^, 

where,  in  practice,  both  the  response  and  the  predictor  variables  may  have 
been  transformed.  There  are  a  number  of  properties  which,  depending  on 
the  investigational  circumstances,  might  be  important  in  choosing  a  design 
for  fitting  such  a  function  (1.1);  fourteen  were  listed  by  Box  and  Draper 
(1975).  One  characteristic  tjiat  distinguishes  a  good  response  surface  design 
is  its  efficient  use  of  the  available  degrees  of  freedom  to  achieve  the  pro¬ 
perties  desired.  In  particular,  the  problem  of  lack  of  fit  merits  special 
consideration.  Whenever  we  fit  a  model  we  must  always  be  concerned  with  the 
possibility  that  some  greater  degree  of  complexity  may  be  needed  than  that 
allowed  for  already.  We  can,  of  course,  employ  a  design  which  contains  more 
runs  and  is  suitable  for  fitting  the  more  complex  model,  but  then  similar 
questions  arise  concerning  that  model ,  and  so  on.  Clearly  we  cannot  guard 
against  all  possibilities.  A  practical  compromise  is  to 

"k 
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1.  employ,  at  each  stage  of  the  experimental  iteration,  a  model  which 
we  hope  will  be  adequate  but  to 

2.  provide,  in  the  accompanying  design,  for  lack  of  fit  checks  that 

are  sensitive  to  the  feared  discrepancies,  and  to 

3.  use,  whenever  possible,  design  arrangements  which  can  be  augmented 
to  form  larger  designs  appropriate  for  fitting  and  checking  the  more  complex 
model,  if  the  need  for  the  latter  is  revealed  by  the  diagnostic  checks. 
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2.  MEASURES  OF  LACK  OF  FIT 


First  order  designs. 

Useful  first  order  designs  are  the  two-level  factorials  and  fractional 

factorials  of  resolution  three  or  more  which  use  (respectively)  all  or  some 
k 

of  the  2  runs  (+1  »+l ,.  ..  ,+l).  As  discussed,  for  example,  !>y  Box  and  Wilson 
(1951),  fractions  can  be  chosei  so  that  checking  functions  are  associated 
with  the  residual  degrees  of  freedom  containing  feared  interactions.  More¬ 
over,  if  we  limit  the  choice  to  designs  of  resolution  at  least  four,  all 
coefficients  in  the  first  order  model  are  rendered  free  of  second  order 
aliases.  Alternatively,  or  in  addition,  (see  De  Baun,  1956),  by  adding  nQ 
center  points  to  any  such  design,  a  contrast  between  the  average  response 
at  the  center  and  the  average  ’"esponse  at  the  factorial  points  is  made  avail 
able.  This  contrast  provides  an  estimate  of  the  sum  (Bj  j+892+.  .  .+8^)  of 
the  pure  quadratic  coefficients,  if  a  second  order  model  involving  these 
coefficients  were,  in  fact,  needed.  It  thus  provides  a  check  for  the  need 
for  second  order  terms  in  the  common  situation  where  we  are  approximating 
a  (possibly  ridgy)  maximum  or  minimum  and  consequently  the  are  either 
of  the  same  sign  or  are  near  to  zero. 

Second  order  designs. 

One  suitable  class  of  second  order  designs  (see  Box  and  Wilson,  1951) 
is  of  the  central  composite  type.  A  design  of  this  kind  for  k  =  3  with  nQ 
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added  center  points  is  defined  by  the  columns  headed  x^ ,  x2,  and  x3  in 
Table  1.  This  design  has  14  +  nQ  degrees  of  freedom  (df)  available.  Ten 
of  these  are  used  for  fitting  the  model. 

There  are  ten  possible  third  order  columns,  namely  those  formed  by 
creating  entries  of  the  following  form  (grouped  as  shown  for  our  convenience 
in  a  moment): 


(*1  ,X1 Xg ,xlx3^  >  (x2,x2xj,x2x3);  ^x3’*3*l  ,X3X2^  ’  X1X2X3* 


(2.1) 


The  last  of  these,  x^x2x3,  forms  a  separate  column  (shown  in  Table  1) 

orthogonal  v.0  the  first  ten.  The  other  nine  of  equation  (2.1)  form  three  sets 

of  three  as  indicated  by  the  parentheses. 

Now  suppose  that  the  elements  of  these  third  order  columns  are  regressed 

on  those  of  the  first  ten  columns  required  to  fit  the  second  order  model,  and 

residuals  are  then  taken  to  provide  the  residual  vectors  x^j  (from  x^), 

2 

x122  (from  xix2^*  and  so  on*  *s  9uic,c1y  seen  that 


xiii  =  ^'a  ^xijj’  1  * 


(2.2) 


Thus,  the  residual  vectors  are  confounded  in  three  sets  of  three.  Further¬ 
more,  x^ ,  x 222  and  x333  are  mutually  orthogonal.  These  vectors,  rescaled 
so  that  elements  of  x...  in  the  cube  portion  of  the  design  are  +l's,  are 
shown  in  Table  1. 

Consider  now  a  column  x.^  in  relation  to  Figure  1,  which  shows  the 


L 
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1 . 

2a2 -6 

it 

M  is  an  nQ  by  (nQ-l)  nonsingular  matrix,  all  of  v.hose  columns  sum  to  zero.  The 
(nQ-l)  columns  provide  any  set  of  pure  error  comparisons  that  might  be  selected. 
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Figure  1. 


Projection  of  design  points  on  the  x-  axis  for  a  cube 
plus  star  plus  center  points  design  in  three  factors; 
the  colunn  provides  (apart  from  a  constant)  an 

estimate  of  the  difference  between  the  two  slopes 
shown.  The  scopes  are  the  same  if  the  true  model 
is  quadratic. 


projection  of  the  points  of  the  composite  design  onto  the  axis.  Denoting 
the  average  of  all  the  responses  at  x.  *  -a,  -1,  1,  a  by  y_a>  y ,  y+j ,  and  y+a, 
respectively,  we  see  that  the  contrast  associated  with  x-^.  is  proportional 
to  (in  an  obvious  vector  notation) 


1  y+  -  y 

ci  =  i  =  — 2“ 


y  -  y 

■'a  J-a 


(2.3) 


Referring  to  Figure  1,  we  see  that  the  first  of  the  expressions  on  the  right 
of  equation  (2.3)  provides  an  'ntuitive  estimate  of  the  slope  of  the  chord 
AB,  while  the  second  is  an  estimate  of  the  slope  of  the  chord  CD.  These 
chords  are  parallel  if  the  response  is  quadratic  when  E(c^)  =  0,  but  otherwise 


Etc,)  - 


(1-q2)B 


in 


3 

I 

ofi 


8- . . 

iJJ 


(2.4) 


and  would  usually  be  non-zero.  Thus,  the  contrast  c^  provides  a  natural  check 
on  the  quadiatic  model  in  the  x..  direction,  for  i  =  1,  2,  and  3. 

In  general,  a  composite  design  is  formed  of: 

(a)  A  "cube",  consisting  of  a  2^  factorial,  or  a  2k-p  fractional  factorial, 
made  up  of  points  of  the  type  f  +  ,+1  •  •  ,+l) »  of  resolution  R  >  5  (Box  and 
Hunter,  1961)  replicated  f(>l)  times.  There  are  thus  nc  =  f2*c~p  such  points 
(where  p  may  be  zero). 

(b)  A  "star",  that  is,  2k  points  (+ct,0,0,. . .  ,0) ,  (0,+a,0,. . .  ,0) . 

(0,0,0,. ..  ,+a)  on  the  predictor  variable  axes,  replicated  r  times,  so  that 
there  are  n$  =  2kr  points  in  all. 

(c)  Center  points  (0,0,..., 0),  nQ  in  number. 

For  any  particular  design,  k  is  given  and  the  values  of  p,  f,  a,  and  nQ  need 
to  be  specified. 


It  is  shown  in  the  Appendix  that,  for  any  such  design,  k  sets  of 

2 

columns  can  be  isolated  with  the  ith  set  containing  the  k  columns  x^, 
j  -  l,2,...,k.  This  ith  set  is  associated  with  a  single  vector  x.^  which 
is  orthogonal  to  the  (k+l)(k+2)/2  columns  required  for  fitting  the  second 
degree  equation  and  is  also  orthogonal  to  the  (k-1)  similar'y  constructed 
vectors  x...,  j  f  i.  Adopting  a  notation  similar  to  that  used  previously 
for  the  k  =  3  case,  we  find  that  the  k  orthogonal  contrasts  defined  by 
these  vectors  are  such  that,  for  i  *  l,2,...,k. 


1 

f2k'p 


U 

2 


(2.5) 


and  may  be  interpreted,  as  before,  as  checks  for  parallelism  of  chords  in 
the  k  different  axial  directions. 

It  may  also  be  shown  that,  on  the  assumption  of  the  need  for  a  cubic 
model , 


E(c.)  *  (1  -ct  )B. . .  +  E 

’  Hi 


$.  .  . 

UJ 


(2.6) 
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3.  RELATIONSHIP  TO  TRANSFORMATION  OF  PREDICTOR  VARIABLES. 


We  mentioned  earlier  the  dilemma  in  which  an  investigator  invariably 
finds  himself  or  herself.  To  run  a  design  appropriate  for  fitting  an  un¬ 
necessarily  elaborate  model  is  wasteful,  but  to  oversimplify  might  be  mis¬ 
leading.  In  particular,  when  we  hope  that  a  second  degree  polynomial  may 
be  adequate  for  representation,  we  shall  wish  to  provide  ourselves  with 
sensible  checks  without  going  to  the  extreme  of  running  a  complete  third  order 
design.  We  need  then  to  check  for  what  we  believe  will  be  the  most  likely 
discrepancies  (if,  indeed,  there  are  any  at  all).  We  think  that,  in  many 
practical  situations,  the  k  "pairs  of  parallel  chords"  checks  which  the 
second  order  composite  design  always  provides,  have  a  natural  appeal. 

The  problem  may  also  be  approached  in  terms  of  the  possible  need  for 
transformation  of  the  predictor  variables.  It  is  natural  to  speculate  that, 
if  a  second  degree  equation  is  not  adequate  for  the  original  uncoded  pre¬ 
dictor  variables,  >$2”  *  *  *^k’  sa^  ^or  coded  versions  ,x2>. . .  .x^) , 

it  might  be  adequate  if  some  transformation  of  the  were  used.  The  idea 
obviously  relates  to  the  earlier  one  because,  in  general,  one  piece  of  evidence 
of  the  need  for  transformation  would  be  the  lack  of  chord  parallelism  in  one 
or  more  axial  directions.  What  we  shall  show  is  that  the  k  contrasts 
Cj  ,C2>. ..  »C|^  associated  with  chord  parallelism  also  contain  information  pro¬ 
vided  by  the  data  about  the  need  for  transformation  of  the  predictor  variables. 
However,  we  shall  also  see  that  the  existence  of  interactions  B-^,  i  f  j,  in 
the  second  order  model  means  that  the  pieces  of  information  provided  by  the 
Cj  overlap  to  some  extent,  that  is  we  cannot  relate  c.  only  to  transformation 
on  x.  when  B^ ^  f  0. 

8  I J 
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Suppose,  for  example,  that  a  composite  design  is  run  in  the  coded 


variables 


»1  ■  <V«io>/sl 


(4.1) 


in  a  situation  where  the  underlying  response  function  f(£)  can  be  approximated 

by  a  second  degree  polynomial  in  the  transformed  original  variables,  namely 
Xi 

.  Expanding  via  a  Taylor's  series  expansion  about  the  values  Aq  =  (1, 
and  cutting  off  at  first  order  terms  provides  the  approximation 

k 

n  =  F(fJ  +  Z  (X, -1  )z-  (4.2) 

i~l  1  1 


where 


L 


and 


3F 
A . 


Xi 

3\. 


X  = 


xo 


3F 

A. 


&!  +  23 1 •  £•  +  Z  g! 
pi  nsi  iJsJ* 


X  =  Xo 


r 


xi 


5 

3A. 


A  =  A. 


-  ho  *  r-ilnF-io  *  bbr- 


10 


.in. 


(4.3) 


(4.4)  ! 


Combining  the  pieces  above,  we  see  that  z^  is  approximated  by  a  cubic  poly¬ 
nomial  in  C,  and  that  the  component  of  z^  orthogonal  to  the  quadratic  model 
is  approximated  by 


vi  =  {3ii4iii  +  I  8ijciiij)/?io* 


(4.7) 


The  elements  £..-u,  u  =  1  »2 . n  are  obtained  by  regressing  £?u  and 

^iu^ju  against  a11  the  variables  1,  €1(J,.  •  •  •  •  ’^ku;^lu^2u”  *  *  ’^k-l  ,u^ku 

which  appear  in  the  quadratic  portion  of  the  model,  and  taking  residuals.  He 

now  rewrite  (4.7)  as 


w-  =  S.{0..x...  +  s-  I  B..x. ..}/£.  . 
l  m  2  ij  iij  io 


(4.8) 


where  £ .  ~  £.Q  +  S.x^ ,  where  additional  terms  of  orders  <_  2  have  been  absorbed 
in  the  quadratic  portion  of  tha  model,  and  where  x.^.  and  x^.j  are  the  standardized 
forms  of  and  ^  employed  earlier.  Also,  B-j  is  the  coefficient  of  the 
term  x-x-  in  the  standardized  variables,  so  that  3  S.S.Bl..  He  now  show 
how  to  apply  these  results  to  composite  designs. 

Example.  Consider  the  estimation  of  the  X's  using  a  composite  design,  as 
described  in  Section  1.  The  appropriate  third  order  orthogonal  polynomials 
for  such  a  design  are  found  to  be 


xiij  =  Vj  ' ex) 


(4.9) 


- - .  1 


where 


rxiuxIu/£ 


'iu 


£/ (f+2ra  ) , 


(4.10) 


and 


xm 


♦x. 


(4.11) 


where 


*  = 


f+2ra4 

f+2ra2 


0(l-a2) 


+  a 


(4.12) 


Each  predictor  variable  in  the  design  can  take  only  five  values, 
namely  ( -a, -1 ,0 , 1 ,a) .  The  values  of  the  third  order  polynomials  at 
existing  combinations  of  these  values  are  as  follows. 


xi 

xi 

xiij 

x. . . 
ill 

xiii 

+1 

+1 

(l-0)xj 

d-e)x. 

ci-aha-ejxj 

+o 

0 

0 

-exi 

(l-a2)(-0)x. 

0 

+a 

-0x. 

0 

0 

1 

0 

0 

0 

0 

0 

2 

Note  that  =  (1-a  )x^j,  which  exhibits  a  well-known  aliasing 
feature  of  composite  designs.  Thus  (4.8)  becomes,  for  this  type 
of  design, 


w.  »  S. (B . .x .  . . 
l  1  11  in 


1  ^  2-1 
7  S  Bh(!-0  Ax. ..}/£. 


10 


(4.13) 


The  possible  need  for  a  transformation  parameter  A^  for  x.  is 
thus  examined  as  follows: 

1.  Fit  a  full  second  order  model  to  the  response  data  and 
so  obtain  estimates  bQ,b^,  b^,  bjj  of  the  corresponding  0's. 

2.  Substitute  b^  and  b^  into  (4.13)  to  obtain  on  estimated 
w^  value;  call  this  w^.  Create  the  w^  vector  by  substituting  the 

A 

design  points  successively  in  w^. 

3.  Regress  the  vector  of  observations  y  on  to  the  vectors 

A. 

wi,  i  ®  l,2,...,k.  This  (see  Eqs .  (4.2)  through  (4.8))  provides 
estimates  (X^-l)  of  the  (A^-l). 

4.  Test,  approximately,  the  reality  or  otherwise  of  the 

A 

(A^-l)  by  the  usual  t-  or  F-test,  pretending  that  w^  is  fixed,  i.e., 
ignoring  the  fact  that  estimates  b^j  and  b-j  are  used  in  it.  Signifi- 

A 

cant  (Xi*l)  indicate  the  need  for  transformations  of  parameter 

A 

values  estimated  by  the  A^. 


Comments 

(a)  The  w^  are  all  linear  combinations  of  ,x222  *  *  *  *  ,xkkk 
Thus,  in  general,  they  are  not  orthogonal  to  one  another,  although 

a 

xiii  an<*  Xjjj  arC:  orthogonal  for  i  ^  j.  This  means  the  are 
correlated  with  one  another,  in  general.  One  way  of  proceeding 
is  to  obtain  the  regression  in  (3)  via  a  stepwise  regression  pro- 

A 

cedure,  and  to  assess  and  test  the  (X^-l)  in  the  order  they  enter 
the  equation. 
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(b)  Eq.  (4.13)  implies  that  a  transformation  in  any  one 
predictor  variable  depends  not  only  on  what  is  happening  in  the 
direction  of  that  predictor  variable  but  also  depends  on  what  is 
happening  in  the  directions  of  the  other  predictors,  in  general, 
if  6 +  0. 

(c)  If  =  0  all  i,  j,  i  f  j  then  (4.13)  implies  that 

the  transformation  needed,  if  any,  is  directly  related  to  the  cubic 

lack  of  fit  The  regression  can  be  performed  directly  on 

x...  in  such  a  case.  This  result  could  be  applied  to  estimate 
-in 

transformations  in  the  canonical  directions. 
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APPENDIX 


We  here  prove  the  results  summarized  in  Section  2.  The  full  cubic  model 
in  k  variables  .x^,. . . is  given  by 


*  =  *o  +  f  Vi  +  =  Vi 


+  Z  E  P^^X.X.  +  E  E  8,‘4^X.X. 


i  <  j 


u  i  j 


i  j 


i  JJ  i  J 


+  Z  £  18.  ..X.X-X. 

i  <  j  <  £  1JA  1  J  4 


(Al) 


where  all  subscripts  run  l,2,...,n  and  the  restrictions  shown  for  them  are 
observed  where  necessary.  The  form  of  the  X  matrix  in  the  regression  model 

-w 

y  =  XB  +  e  when  &e  design  consists  of  f  "cubes"  (f  replicates  of  a  2k_p 
factorial  design  with  coordinates  of  form  (+1 ,+l . .+1))  plus  r  "stars" 

(r  replicates  of  2k  axial  points  distance  a  from  the  origin)  plus  nQ  center 
points  is  as  shown  in  Table  2.  We  can  denote  columns  by  placing  square  brackets 
around  the  column  head;  for  example  [x^]  will  denote  the  x^  column,  and  so  on. 

We  write 


F  = 


f2k-p 


for  the  number  of  factorial  points. 
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Table  2.  The  X  Matrix  For  a  Cubic  Model 


All  of  the  cubic  columns  are  orthogonal  to  all  of  the  other  columns 

3  2 

with  the  following  exceptions:  [xf]  is  not  orthogonal  to  [x^],  nor  to  [x.x.]; 

I  '  ^  w 

7  3  2 

[x.x.]  is  not  orthogonal  to  [x.],  nor  to  [x<],  nor  to  [x,.x.].  The  first  step 

1  J  *  1  ■  * 

3  2 

is  to  regress  the  [x..]  and  C x - x ^ 3  vectors  on  the  [x^]  and  take  residuals. 

Because  the  columns  involved  are  orthogonal  to  [x  ],  no  adjustment  for  means 

2 

is  needed.  We  denote  the  "cube  portion"  of  the  [x . ]  and  [x^]  vectors  by  c-» 
as  indicated  in  the  table.  These  two  sets  of  residuals  are,  where  the  prime 
denotes  transpose. 


[X?l  -  tcx^  3 ■ , 3 • [X, 3> Cxf 1 
and 

[x^j]  -  {[x1 3 ’  Cx1  Xj3/Cxi ]  •  [xi 3) 3 


o 

both  of  which  reduce  to  multiples  (1-a  )m  and  m,  respectively,  where 
m  =  2a2r/(F+2a2r) ,  of  the  same  vector  [x...],  say.  For  example, 

=  ,d,-d,d,-d. . .0,0,. . . ,0]’ , 


where 


d  =  F/(2ar) , 

and  where  there  are  r  sets  of  (d,-d's)  in  the  vector.  In  general,  for  [x..^]', 
cJ  will  be  replaced  by  c!  and  the  position  of  the  id's  will  correspond  to  those 
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of  the  +a’s  in  the  corresponding  [x^]'  vector.  Note  that,  because 
cJCj-O.ffj.ltls  obvious  that  [x^]  and  [x^]  are  orthogonal. 

It  follows  that  the  k  cubic  coefficients  8^,  6^-  (jfl.j-1.2 . 1 

otherwise)  cannot  be  estimated  individually  but  only  in  linear  combination, 
and  that  the  appropriate  estimating  function  for  this  is 


=  c.y,  +  d(-rya+ry_a) 

where  y^  is  the  portion  of  y  corresponding  to  the  cube  part  of  the  design, 
and  y  ,  y_Q  are,  respectively  the  averages  of  observations  taken  at  the  a 
and  -a  axial  points  on  the  axis.  If  we  similarly  denote  by  y+  and  y  the 
averages  of  the  F/2  observations  in  y1  corresponding  to  1  and  -1  in  c^, 
respectively,  it  follows  that 


-1 


f“  £iii  =  (y+-y_)/2  -  (ya-y.a)/(Za), 


The  expected  value  of  F"  is 


where  X  is  as  in  Table  2  and  the  coefficients  of  8  correspond  to  the 
columns  in  the  obvious  manner,  namely 
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2 

Because  [x....]  is  orthogonal  to  all  columns  of  X  except  the  [x.]  and 

[x.x2]  columns,  it  follows  that 
1  J 


E(F  =  F  1{Si£i"2m3d}6iii  +  F  ^iSi  ^  eijj 


*  (1-a  )B^j  +  £  4 ^ - 


J+i 


•  jj  * 


-Thus  all  the  stated  results  have  been  established. 
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